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Abstract 



In ftn evaluation study, employing a pretest-posttest control 
group design, 13 elementary schools '«re paired based on character- 
istics of their student bodies, faculties, principals, and special 
programs. One school from each pair was randomly assigned to experi- 
mental and control conditions. Fourteen third grade teachers In the 
six experimental schools were provided with In-service training In 
the Hagic Circle Kimary prevention strategy. The teachers con- 
ducted Magic Circle In their classrooms with 217 students over the 
course of a school year. Fourteen third grade teachers and 250 
students In the seven control schools served as a comparison group 
and received no Interventions. 

, » 

The Magic circle In-service training consisted of ten, weekly, 
two-hour, after-school sessions, supplemented by a mlnlmtmi of four 
classroon visits by the trainer to observe and guide teacher Imple- 
mentation of "Circles." 

.process evaluation data included a) documentation of In-sejrvlce 
training by outside observers, b) teacher feedback on the Individual 
training sessions, c) questionnaire surveys of teachers at the end of 
training and the end of the school year, d) Interviews with a sample of 
teachers and principals, e) observations of classroom Implementation, 
and f) weekly reports by teachers about classroom Implementation. 
Process evaluation showed that the training was very highly regarded 
by the teachers, that all the teachers mastered most of the skills 
taught In the training, and that they found most of the skills to be 
quite useful 5n the classroom. The number of Circles that teachers 
conducted In their classrooms varied considerably, ranglog from a 
total of 8 per student to 36 per student, and averaging Z2 per student. 

Experimental and control teachers and students were pre- and post- 
tested, and data regarding achievement and attendance were gathered 
from school district records. Teachers also rated their students' 
classroom behavior. 

Post-test comparisons of experimental and control group teachers 
showed that experimental teachers were significantly more satisfied 
with teaching than controls, and were using teaching skills related 
to the training significantly more often. 

Student- and class-level analyses of variance and covarlance on 
outcome data showed that compared with controls, boys In the experi- 
mental group had slgr^lflcantly higher social self-esteem at the end 
of the school year. Also, experimental boys were rated by their 
teachers as presenting more minor but fewer major discipline problems. 
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No effects of Magic Circle were obtained for girls. Furthermore, the 
level of exposure to Magic Circle was not related to experimental 
students* post-test outcomes. 
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Magic Circle Is the popular name foi^ the Human Develofwient Program 
developed by Palomares, Bessiel , and Ball (Ball, Note 1). The program 
was devised as a primary prevention strate'gy for positively influencing 
students' social and emotional development. According to Its creators, 
expected ef fiects of the program range from Improved coimiuni cation and 

* 

listening skills to Improved self-conce^an^ peer relationships, 

Magic Circle Is used widely; over'9,000 Individuals have been trained 

to conduct "Circles" since 1977 (Ward, Note 2). 

Magic Circle Is a curriculum for conducting small group discussions 

In the classroom. In a Circle session, which typlCcUy lasts 15-30 

minutes, the teacher leads a aroup of 5-12 students through-^ discussion 

. ■ ■ 

of a particular topic. The teacher employs. methods which encourage 
the students' expression of thoughts and feelings without running many 
of the risks of other group Interaction formats. Participation Is 
always voluntary. Confrontation and debate are ne.yer allowed."' Although 

students may be asked about their feelings, e)ttended probing and analysis 

. - . \^ 

of Individual comments are strictly forbidden. . Th^ key elements of the 
Circle are that everyone gets an opportunity to contribute— even If they 
choose to remain silent— and that each contributor is listened to 
carefully by. the other Circle menders. 

nun^er of unpublished outcome evaluations of Magic Circle have 
been conducted since 1970. Most of the available research has been 
summarized by the Human Development Training Institute (Note 3), the 



developers of the program. Tw6nty-one of the studies sunmarized In 
the HDTI review employed experimental or non-equivalent control groups. 
In these studies ^-according to the HDTI review, preschool through grade 
7 students participated in 5-150 Circle sessions. On the average^ 
students attended more than three sessions per week (M» 3,^, SD" 1.5) 
over a 19 week period (M « 18,7, SD» 9.3), representing participation 
in a total of 60 Circle sessions (M = 60;4, $0- 51,4). 

The student outcomes examined In the 21 studies Included measures 
of five variables: self-concept, peer relationships, locus of control, 
reading perfonnatice, and student behavior. Twelve of these studies 
Investigated the effect of Magic Circle upon a single variable. Only 
three studies assessed effects on- three or more variables. Of the 
12 studies that examined the effect of Magic Circle upon student self-" 
concept, nine found no evi'dence for an effect* and three found positive 
effects for sorae experimental groups but not others. Four studies 
measured effects on peer relationships. Three of these found positive 
effects and the fourth found a mixed negative effect. Four studies 
measured locus of control, 'or students' feelings of responsibility 
for their own behavior. One of these found evidence of a positive 
effect. Seven studies measured effects on reading performance. 
Four showed positive effects, one showed mixed effects ; and two showed 
negative or no effects. Of the seven studies In which teachers rated 
studer.is' behavior, four showed positive effects, and three found no 
effect. 

These patterns of results provide some evidence that Magic Circle 
can Improve students' peer re1dt1onsh1pS| reading performance* class- 



room behavior, and» perhaps, self-concept and locus oY control* These 
results should be Interpreted cautiously, however, since the research 
summaries were written by the program developers, and, more importantly, 
since the research suffers from several methoaological shortcomings. 
First, previous studies have generally failed to examine systematically 
^fagic Circle's impact upon the range of variables of interest,; most 
studies measured only one or two variables. Better understanding of 
Magic Circle's effects would follow from studies which examine the 
various relationships among the important outcome variables. 

Second, In many Instances, the measures selected to operatlon- 
allze outcome variables have not been specific to Magic Circle objec- 
tives. For example, general self-concept measures have been widely 
utilised, insteadof measures more sensitive to changes in social 
self-concept. 

Third, most outcome studies have not been accompanied by adequate ; 
process evaluation* Process evaluation provides data on the treatment 
Implementation, e.g* , the quality as well as the quantity of Circles * 
conducted In each classroom* Such Information makes possible investi* ' 
gatlons cf process-outcome relationships and enables better identlfi* 
cation of Important treatment aspects* 

The present study attempts to address the shortcomings described * 
above by Coupling comprehensive outcome and process evaluations of 
a Magic Circle teacher in-service training program. The Magic Circle- 
training was provided in the first year of a three-year program of Inter* 
^ventions to be delivered to a cohort of students as part of an ongoing 
effort to Implement and evaluate promising primary prevention strategies* 



The present evaluation assessed the Impact of Magic Circle on the 

following student outcome variables: social self-esteem, atti tildes 

toward peers, attitudes toward teacher,. academic self-esteem, locus 

of control, attitudes toward schoo} , perception of peers' attitudes 

toward school academic achievement, school attendance, and behavior 

problems* Among these variables, we hypothesized that Magic Circle ^ 

-would be most likely to impact social self-esteem and attitudes toward 

peers, because the strategy most directly bears on these variables and, 

because they have been impacted in previous research* Positive effects 

upon the. other variables were also anticipated, although Magic Circle 

* 

does not address these as directly* 

Teacher outcomes examined in the evaluation included the importance 
of and effectiveness at implementing Magic Circle objectives, the use 
of Magic Circle skills In the cVassroom, personal satisfaction with 
teaching, and faculty cohesiveness. We expected that Magic Circle in- 
service training would positively affect these teacher outcomes with 
the^exception of faculty cohesiveness (because the training was only 
provided to the grade 3 teachers at each experimental school ) ^ 



Method 

Assignment of Schools to Condition 

Thirteen elementary schools (grades K*6) from a predominantly 
white, middle-class, suburban public school system in Northern Calif- 
ornia were paired based on characteristics of their students, faculties, 
principals and special programs. One school from each pair was randomly 
assigned to the experimental condition and the other to the control 
condition. 

Table 1 shows, for the experimental and control conditions, 
the school means and standard deviations of the measures employed 
in matching the schools. For each school, these measures included: 

a) total student enrollment in the spring priur to the study, 

b) the number of students in the cohort group during the prion spring; 

c) the number of "pure" grade 3 (the cohort grade) classes and the 
number of -"mixed" classes containing grade 3 students; d) average total 

.reading scores for the cohort group on the Stanford Achievement 
Test administered during the prior spring; e) combined reading com- 
prehension and vocabulary scores for the cohort' group oo the Stanford 
, Diagnostic Reading Test, administered at the beginning of the study; 
f) the percentage of students from families receiving Aid for Families 
with Dependent Children (AFDC); g) the percentage of students classified 



^Since an odd number of schools existed, one triplet was formed 
from which one school vyas randomly a§;si'gned to the experimental con- 
dition and the other two to the tOn^rol condition. 



Table 1- 

comparison of experimental and control schools^after 
matching and random assignment to condition 

Experimental Schools (N-6) Control Schools (N-7) 

School Standard School Standard 

Mean Deviation Mean Deviation 



Total School Enrollment 




340 


'1 30 


294 


49 


Cohort Grade Enrollment 




51.0 


24.5 


40.3 


11 .7 


Number of Cohort Classes per- Scho 
Grade 3, only classes 


ol 


2.00 


0.63 


1.29 


0.95 


"Mixed" grade classes 




0,33 


0 52 


0.57 ~ 


0 98 


SAT Reading Achievement Scores 




1 15.2 


7.8 


1 1/ ;o 


iZ.C 


SDRT Reading Comprehension 
and Vocabulary Scores 




66.0 


4.6 


65.0 


6.0 


Percent of Enrollment Receiving AFDC 


13,0% 


8.0% 


11.6% ■ 


10.4% 


Percent of Spanish-Speaking 




2.'3% 


3.2% 


1.5% 


3.1% 


Average NunOier-of. Unexcused 
Absences per Pupil 




2.0 


0.3 


1.8 


0.9 


Socioeconomic Status Rating 




46,7 


34.2 


47 .1." 


sb.i 


Eligible Faculty Support Rating 




0.86 


0.22 


0.93 


0.19 


Eligible Faculty Competence Rating 


0.«5 


0.39 


0.60 


0.45 


School Principal Support Rating 




0,75 


0.42 


0.71 


0.39 


School Principal Coitipetence Rating 


' 0,66 


0,41 


0.29 


0,39 



o 10 
ERIC ... 



as Spanish-speaktng with either" limited or no English language skills; 
h) the average number of unexcused absences per student during the 
prior school year; i) an estimate of the student body's socioeconomic 
status (made by a school district official). 

Iri addition, two district officials rated each school regard- i 
ing the degree to which': a) elig-tble teachers would support and par- 
ticipate%in the in-service training; b) eligible teachers were already 
competent in classroom management and Interpersonal skills; c) the 
principal would support the in-service training; an<| d) the princi- 
pal had influence over his or her^teaching staff. 
Subjects ' : 

Initially, all grade 3 students and their teachers in the 

six experimental schools constituted the experimental group* One 

class of students and Its teacher were dropped from the study because 

the teacher left the Magic Circle In-service training after the third 

session and never conducted Magic Circles in her classroom. The 

* 

other 1-* grade 3 teachers completed the in-service training and imple- 
mented Magic Circle in their classrooms. The control group consisted 
of all grade 3 students and their teachers in the seven control 
■schools. There were 14 classrooms in each condition: seven pure 
grade 3 classes and seven mixed -classes. 

At the beginning of the study, 251 third graders were enrolled 
in the experimental classes and 281 were enrolled in the control 
classes. Thirty-four students v/ere excluded from the experimental 
condition and 31 from the control condition due to lack of parental 
permission for testing or because they transferred out of the school 



\ 

during the year. The experimental group consisted of 102 boys and 
115 girls, and the control group consisted of 131 boys and 119 girls. 
The ethnic conposition the sample was 86% {N 403) White and 14?£ 
(N^=64) minority, with Mexican-Americans comprisi^ig the largest single 
minority group (N "29). 
In-Service Training Program 

. Ten weekly N;w6*hour sessions of Magic Circle in-service 
training were conducted by a highly experienced teacher trainer. 
The curriculum included both the basic skills for conducting Magic 
Circles and supplementary skills for improving the quality of teach- 
student interactions within the Circles and in other classroom 
activities. The 14 experimental teachers who attended the training 
sessions and implemented Magic Circle in their classrocns were paid a 
$200 stipend and were offered graduate-level credits through a local 
university. 

In the first j^everal training sessions, the trarner introduced 
the teachers to the basic skills necessary for conducting Magic Circle 
discussions. The trainer showed a videotape of a Magic Circle, led 
Magic Circles in which the teachers participated, and gave each teacher 
the opportunity to lead prac4:ice Circles. Beginning with the second 
session, the trainer conducted classroom activity reviews in which 
the teachers discussed their experiences applying the skills and 
activities during the prior week. 

The next several sessions stressed skill development in three 
areas: leading the Circle, enforcing Circle rul&s, and dealjng with 



children who share inappropriate or personal information in the 

Circle. Two types of Magic Circle discussion topics were introduced: 

**awareness'* and "mastery" topics. At the beginning of each session, 

the trainer introduced a self -concept enhancement activity which 

the teachers coulTimplement in their classrooms. 

The last several sessions focused on dealing with special 

* 

problems that individual students present (e.g.^ the shy child, 
the child who answers Irrelevantly). In these sessions, the trainer 
provided instruction in listening and other conimunication skills. 
One of the cojranunicatlon skills was ''reflecting feelings,'* a tech- 
nique designed to identify for speakers the feeling they have conveyed. 
Teachers were introduced to "social interaction'* discussion topics; 
they learned how to generate useful Ideas via "brainstorming**; and 
they learned how to give specific or *'tailored*' feedback to students. 

During the school year^ the trainer visited each of the partici- 
pant teachers* classrooms a minimum of four times. During some of the 
initial visits, the trainer led Magic Circles or demonstrated other 
skills for the teacher. During most of the visits^ the trainer observed 
the teacher conducting a Circle and later provided feedback. On these 
occasions, immediately afteiT" leaving the classroom^ the trainer com- 
pleted a process evaluation form based upon notes taken during the 
observation. 

I nstrumentation and Measures 

Process evaluation data. Several methods, were , used to monitor 
implementation of Magic Circle, and to ascertain reactions to the Circle 
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training. These methods included a) detailed records of all in-service 
teacher training sessions, prepared by an expert "outside" observer, 
b) records of teacher attendance at training sessions, c) surveying 
of teachers after each training session to obtain feedback on the 
sessions, d) ^individual interviews with selected teachers and princi- 
pals to obtain more detailed information on reactions to the training, 
e) questionnaire surveys of teachers after training and again at the 
end of the school year to monitor implementation and to assess the 
adequacy of the training, f) classroom observations by the in-service 
trainer of the teachers while conducting Circles with their students* 
g) weekly reports from teachers on how often they conducted Circles 
in their classrooms, and h) interviews with selected control school 
teachers and ineligible experimental school teachers to identify aiiy 
diffusion of the training. These methods and the process evaluation 
data gathered with them are described in the next major Section of 
the paper. 

Student self-report outcome data . Student pretest and post-test 
self -report data were obtained with the Self Observation Scales 
(Intermediate Level, Form C) developed by Stenner and Katzenmeyer 
(Note 4). This nationally nomed instrument h^ts empirically deter- 
mined scales which measure the ways children perceive themselves 
and^eir relationships to their peers, their teacher* and their school. 
The questionnaire contains 60 statements to which students respond 
"yeii" or "no." 




On the post-test, data were obtained from students with 
the Student Questionnaire (ELE-2), an instrument developed for this 
study* This instrument consists of two sections* Part 1 contains 
18 items selected from the Intellectual AchievGment Responsibility 
Questionnaire developed by Crandall* Katkovsky, and Crandall (1965)* 
Selection of items was based upon published psychometric properties. 
Four items referring to "parents" were adapted to read *'an adult 
who knows you" In order to conform to California Education Code. 
These items measure the belief in one's own control over, and responsi- 
bility for, intellectual-academic successes and failures* 

Part 2 of the Student Questionnaire contains three sets of 
items: a) the Scholastic scale from the Intermediate-Level of 
the Self Appraisal Inventory (20 itans) developed by the Instructional 
Objectives Exchange (1972b), a criterion-referenced measure of academic 
self-esteem; b) the Authority and Control (12 items) and Interpersonal 
Relationships With Pupils (12 items) scales from the Intermediate- 
Level of the School Sentiment Index developed by the Instructional 
Objectives Exchange (1972a), a criterion-referenced measure of atti* 
tudes toward school; and d) a measure of perceived peer attitudes 
toward school developed for'this study by adapting 11 items from 
eight instruments that measure attitudes toward school* 

The measures employed in the data analysis for the present 
study were derived from a theoretical model and empirical scaling 
analyses. The details the scaling procedures and results have been 



reported by Moskowitz, Condon, Brewer, Schaps, and Malvin {Note 5). 
Pretest and post-test data were collected from large samples of 
students 1n grades 4-6 in addition to the third grade students in 
this study. Random samples were selected from each grade level. 
Item means, variances, and intercorrelations were found to be similar 
across grade levels; thus, scaling procedures were applied across 
grades. Item intercorrelations from the pretest and from the Post- 
test data were subjected to separate multiple group confirmatory 
factor analyses .2 The resultant pretest and post- test scales 
appe.r in Table 2 with the number of- items contained 1n each, and 
their internal consistency reliabilities estimated by coefficient 
alpha. The final scales Included pretest and post-test measures 
of social self-esteem, attitudes toward peers, affective teaching 
climate, ana attitudes toward school. Other post-test measures ' 
included academic self-esteem, locus of control for success, locus 

of control for failure, and perceived peer attitudes toward school. 

\ ,' 

^ Student archival outcome data . In addition to the self-r^port 
data gathered directly from students, we obtained achievemenl^/lata 
collected by the school district as part of its regular testing 

activities. For cur pretest measure (Prior Read) we employe^ reading 
comprehension stanine scores from, the Stanford Diagnostic Reading Test 

{Level Green) administered in September, 1978. For the 10% of students 
who missed this test, we substituted total reading stanine scores from 

the Stanford Achievement Test (Primary Level 11) administered the prior May. 

^The matrices were computed using pair-wise deletion of missing 
data and conwunalities were inserted into their diagonal elements. * 
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Table 2 








STUDENT PRETEST ANO POST-TEST SCALES, NUMBER OF ITEMS ANO 
INTERNAL CONSISTENCY RELIABILITIES {COEFFICIENT ALPHA) 






PRETEST^ 


POST-TEST^ 


iubscale 


Number of 
Items 


Reliability 


Number of 
Items 


Reliability 


)0C7al 5eit-tSteein 
(Social Self) 


- 6 


.62 


6 


. .66 


\ttitudes Toward Peers 
{Att Peers) 


8 


.76 


8 


.80 


\ffective Teaching Clifnate 
(Affec Climate) 


7 


.74 




.91 


\cademic Self-Esteem 
(Acad Seff) 


NA*^ 




11 


.79 

1 


.ocus of Control: Success 
(Control Sue) 


NA 




7 


' .56 


.ocus of Control: Failure 
AControl Fail) 


NA 




7 


.62. 


\ttitudes Toward School 
(Att School) 


6 ' 


.68 


6 


.74 , 


'erceived Peer Attitudes Toward School 
(Peer Att Sch) 


NA 




8- , 


.72 ^" 



N » 517 
V = 513 



NA indicates that this scale was not administered. 



According to the test publisher (Karlsen, Madden, and Gardner, 1976), 

I 

these two tests correlate about .87. For our post-test measures (Read 
Math), we employed the total reading and total mathematics raw scores 
\frm the Stanford Achievement Test, Primt y Level III .{Madden, Gardner, 
Rudman, Karlsen, and Merwin, 1973), administered in May, 1979. 

^, Two different measures of student attendance were taken. The 
tiital number of unexcused absences for the second semester (Unex 
Abs) was obtained for each student from the school district's records. 
This type of absence occurs when a student is absisnt from school and 
does not provide the school with a parental excuse indicdting^ttiiat 
the student was sick. In addition,- the average monthly number of 
absences for e?ich student (Total Abs) during the fifth through eighth 
school months (January througlj April) was. extracted from teachers* 
records by-project staff. 

Students^ sex and ethnicity v^ere determined from school district 
records . ,r 

Teacher outcome data . Teacher pretest and post-test self-report 
data were obtained with different versions of the Teacher Question* 
naire, an instrument developed for this study. This instrument 
included measures of a) teacher satisfaction adapted from the Purdue 

Teacher Morale Inventory (Rempel and Bentley, l,9fi4); b) faculty 
cohesiveness adapted from the Teacher Cooperation scale of the 
Teacher Attitude and Classroom Climate Questionnaire (Kaufman, 
Senuiel and Agard, Mote b) and from the Ihtimacy scale of the 
Organization Climate Description. Questionnaire {^^alpin and Croft, 1963); 
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and c) the importance and effectiveness of achieving teaching objec- 
tives related to Magic Circle, a medsure developed for this study. The 

\ 

post-test alsoyncluded a measure of Magic Circle skill implementa- 
tion. This measure was developed for the study and consisted of an 
inventory of classroom behaviors relevant to the Magic Circle in-service 
training. The number of items included in the pretest and po^t-test 
scales and their internal consistency reliabilities estimated 
by coefficient alpha are shown in Table 3 for grade 3-6 teachers\ 

Teacher ratings of student misbehavior were obtained at the 
post-test with the Student Behavior Report.^ This instrument Con- 
tained the class roster for each teacher and asked the teacher to 
indicate how frequently each child had been minor (Minor) andmajor 
(Major) discipline problems during the past four months (January 
through April) on a five-point scale ranging from "never" to "about 
once a day or more/* _ 
Data Collection Procedures 

Student survey , the student pretest was adhiinistered in Octo- 
tier, 1978, and the post^testMn May, 1^9, by six carefully trained " 



substitute teachers. The Self Observation Scales were administered, 
at both times. In addition, the Student Questionnaire was administered 
at the post-test. The questionnaires were. administered in a single 
session in the students' classrooms. For the pretest, the items were 
administered orally by the administrators. For £he post-test, the 
students read the items to themselves {assisted by the administrator 
when necessary). Two make up sessions were held for students who 
were absent from the original session. 
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Table 3 




i 
1 

\| 


t 

TEACHER PRETEST AND POST-TEST SCALES, NUMBER OF ITEMS, AND 
'INTERNAL CONSISTENCY RELIABILITIES (COEFFICIENT ALPHA) 

* 






PRETEST^ 


POST. 


•TEST* 




Number of 


Number of 
ixetns 


Reliability, 


Magic Circle Objectives 
(mc Object) 




o 


7ft 


Teacher Satisfaction 

leacner pans^ ^ 


" " 3 ,68 


8 


.80 


Faculty Cohesiveneiss 
(Faculty Cohes) ■ ' 


' 7- .87 


7 


,90 ' . ; 


• Magic CiVdle Skill Implementation 
(HC Skill) 


NA^ . 


16 


.84^ 


' '^N '= 73 








h i 60' 

1 JIA.JiadlcateS: that this scale was . 








not administet*ed^--^7--r.^^. L..-^.^^ 




/ 




( 






> 
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Teacher survey . The teacher. pretest v/as administered in 
September, 1978, during meetings at each school conducted by - 
project staff. Teachers completed the post-test questionnaires 
in their classrooms while their students were being surveyed. To 
ensure the privacy of their responses, teachers were provided 
with questionnaires containing unique identifiers. 
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" Process Evaluation Methods and Findings 

Descriptions of In-Service Training Sessions 

To document the in-service training, a skilled observer attended 
all training sessions and made detailed notes about both the trainer's 
and the participating teachers' activities. The observer later pre- 
pared lengthy descriptions of a) the agenda as planned and as-actually 
followed, b) any materials used or distributed, c) the trainer's 
presentations, and d) the trainer's and teachers' verbal and nonverbal 
interactions* 

The observer sat In the back of the room and avoided Interactions 
with the teachers. The trainer explained that the observer was present 
to "take notes so that we can develop a training manual." 

These observations showed that the trainer was able to gain the . 
trust and cdnfldencfe of the teachers, who talked frankly about their 
problems and successes in implementing Magic Circle techniques*. The 
teachers seem&d to. ejijoy leamlng^theJD.Ircle skills, to-take the tr^ln-— 
Ing seriously and to participate actively, and to appreciate the emotional 
support and encouragement that the group provided. 
Teacher Attendance at Training Sessions 

The teachers were required to attend nine of the ten in-service . 
sessions in order to receive a $200 stipend. One of the 15 participating 
teachers dropped out after the third session because she was opposed to 
Magic Circle's goals and methods. All of the other 14 teachers completed 
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the training and attended nine or more sessions. Six of these teachers 
also attended a "reunion" session held two months after the end of training 

Principals from the expenmental schools were also Invited, to attend 
the training, and all of the six eligible principals attended one or tnore 
(up to six) sessions. 

Teacher Feedback on the Individual Training Sessions 

At the end of each training session, teachers were asked to 

complete a one-page "feedback form" anonymously. This form solicited 

ratings of, and cownents about, the sessions. On the forrn^ teachers 

rated each session for interest, organization, usefulness, and enjoy- 

ableness, using five-point scales with higher numbers signifying more 

positive ratings. Ratings for the individu&l sessions' all averaged 

4.25 or above on each of the four measures, and 26 of the 40 average 
session ratings were at or above 4.70, indicating that the training 

was very favorably regarded by the teacbers. This high regard was 

also apparent from the teachers' very complimentary written comments, 

and from the almost complete absence of critical comments* 

Interviews with Participating Teachers ar.d Principals 

three experimental school principals and 4 of the 14 participating 

Cj 

teachers were Interviewed individually by a project researcher in 
April, 7979, approximately three months after the training had ended. 
Principals and teachers were randomly selected for the interviews, 
which were semi-structured and open-ended. The interviews were tape- 
recorded witK the respondent's permission, and lasted 15-30 minutes* 



In the interviews, all of the principals and teachers praised 
the training extensively. All felt the training was useful, enjoyable, 
well orgaaized, and relevant to their needs. Three of four teachers 
thought that the trainer's classroom follow-up visits were very helpful 
and supportive; the fourth said the visits were pleasant but unnecessary 
because she was having no difficulty implementing Magic Circle skills. 
All of the teachers "said that the training fully prepared them to use 
Magic Circle effectively, 

• The Interviews revealed that certain in-service skills were used 
less frequently than others^ most notably, a) brainstorming techniques 
and b) teaching students to lead Circle discussions. They also indi- 
cated ^that the' training may have been too lengthy; several principals 
and teachers thought that the last few sessions were redundant and 
ijhnecessary. 

V All four teacfiers reported that their-students thoroughly enjoyed^ 
participating in Circles, and looked forv/ard to them. All reported 
positive effects on- students, both in and out of Circles, including ^ 
increased cooperation and respect; less teasing and criticizing, ^ 
heightened'awareness of others' feelings, and increased feelings of 
community in the classroom. All of the teachers planned to use Magic 
Circle the following year. 
Questionnaire Surveys of Participating Teachers 

At the completion of training (January, 1979), and again near the 
end of the school year (May, 1979), participating teachers provided 
detailed information regarding implementation by completing lengthy 
questionnaires. On the questionnaires, the teachers reported how often 



they used each in-service skill, rated the usefulness of each skill, 
and rated their own mastery of each skill. 

Table 4 shows year«end data on frequency of implementation^, and 
both mid«year and year-end data on usefulness and mastery of the skills. 
The first five sjkills listed in Table 4 are used only when conducting 
Circles, the second set of five skills can be used in Circles or In 
general teaching, and'the last "skill"— self-concept enhancing activities 
is a series of classroom exercises used to supplement the basic Circle 
activity* 

The data regarding frequency of implementation in Table ^4 indicated 
that at year-end, the teachers were using nearly all of the skills at 
least several times per month, and many of the skills at least several 
times per week. However, two skills were not used this frequently: 
preparing students to lead Magic Circles, and brainstorming.— This imple* 
mentation pattern is consistent with the interview data reported above, 
and suggests that the teachers were continuing to use both the Circle* 
specific skills and the "general" skills several months after. the 
training had been completed. 

With regard to the usefulness of the skills. Table 4 shoWs that 
with the exceptions-of student-led Circles and brainstorming, tRe 

e 

teachers rated all of the skills as highly useful at both mid-year and 
year^end* The teachers' average ratings of their own mastery of the 
skills clustered in the araa between "good" and "excellent,*' although 
several were a bit lower* Mastery ratings at year-end were generally 
similar to those at mid-year. 
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TEACHERS' REPORTS ON FREQUENCY, UTILITY, AND QUALITY OF SKILL IMPLEMENTATION AT MID-YEAR AND EMU OF YEAR 



evi 



Cljasspoonr" 
Teaching Skill 

Intr:oduc1ng, explaining and 
restating the topic 

Enforcing the four rules in 
conducting the discussion 

Conducting the cognition 
sta^e and ending, the circle 

Dealing with shy, copying, 
irrelevant answering children 

Preparing students to lead 
Magic Circle 



Leading brainstorming 
activities ' 

Dealing with disruptive 
20 students 

\ Using l-messages 

Using selected parroting 

Using descriptive, non- 
juqgmental terminology 



H End of School Year, 
Percentage of Teachers 
Using the Skill at Least 
.Several Times Per: 



Using self-concept enhancing 
activities 



Month 
92 % 
92 
92 
92 
33 

50 

90 
100 
100 

100 
100 



Week 
58 % 
58 
50 
50 
0 

17 

70 
75 
83 

91 
67 



Mean Ratings of Value 
of the Skill {l«High 
Utility;5=Low Utility) 



Mid-Year 
1.'l5 
1.30 
1.23 ' 
r.33 
3.36 

2.18 

1.46 
1.31 
1.92 

1.31 
1.23 



End-Year 
1.17 

1.17 
1.17 
1.75 

3-00 

, 2.55 

1.42 
1.33 
1.42 

1.25 ' 
1.27 



Mean Ratings of Mastery 
of Skill'^tl^Excellent; 
4=Poop) 



Mid- Year 
1.69 
1.46 
1.92 
2.46 
3.25 

2.29 

1.92 . 

1.46 

1.85 

2.15 
2.15 



End-Year 
1.67 
1.58 ' 
^.08 
2.33 
2.00,' * 

1.88 

1.67 
1.83 
1.73 

2.00 * 
2.00 
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^hese retings completed only by teachers who have used the ?kill in their classrcom- 



To g^ither additional feedback on the in-service training, the 
mid-year questionnaire also asked the teachers to evaluate the in- 
service coufse as a whole, using five-point rating scales. These 
ratings were very favorable with respect to interest _(M = <^.84), 
organization (M = 4.92), usefulness (M = 4.69), and enjoyableness 
(M = 4.92). The trainer was also highly rated along a number of 
dimensions. On the year-end questionnaire, the trainer's classroom 
follow-up visits received a 4.36 average rating for usefulness. 

To sunuiarize, data from the two questionnaire surveys indicate 
that teachers were using most of the skills orr a regular bas'i's, were 
finding the skills useful, considered themselves adept at using the 
skills; and positively evaluated the training, and the trainer. 

Trainer Observations of tlassroom Implementation 

The Magic Circle trainer visited each classroom several times 
during the school year to observe the teachers leading Magic Circles. 
After each classroom observation, the trainer reported a) the number 
of times each skill was used, b) the quality of implementation of each 
skill, and c) the number of occasions on which each skill could have 
been used appropriately but wasn''t (called "missed occasions"). 

Averaged across visits to each classroom, the observational data 
regarding quality of implementation indicate that all 14 teachers were 
using the basic Magic Circle s'-Mls with at least moderate proficiency 



With respect to these basic skills^ the trainer's ratings of the 
teachers* quality of implementation ranged between 3.34 and 5.00 on 
a five-point scale> with 11 of the 14 teachers receiving ratings of 
4.00 or better; The average rating was 4.43 (SO = .45). 

Across all in-service skills (i,e,> both the specific and general 
skins)^ the average quality rating given each teacher by the trainer 
ranged between 3.84 and 4.88 (M - 4.41, SO - .26), Thus, in the 
trainer's estimation, all 14 teachers were also implementing the general'^ 
skills with good or excellent proficiency. 

Several of the skills were only infrequently observed by the 
trainer. These were a) preparing students to lead C^ircles, b) dealing 
with shy-copying-irrelevant answering children, and c) brainstorming. 
No more than four teachers were observed using any of these skills, and 
no teacher was observed using these skills more than an average of 
once per visit. In contrast, at least nineVjteachers were observed 
using each of the ot^er skills, and with substantially higher fre- 
quencies. ^ 

For most of the skills, the trainer noted few mir^ed occasions 
relative to the observed frequencies of implementation. Thus, in the 
trainer's estimation, the teachers were applying the skills in most 
of the appropriate instances. 

In suiranary, data obtained from the trainer's classroom observa- 
tions confirmed that the teachers were using most {but not all) of 
the in-service skills, that they were using the skills proficiently, 
and that they were taking advantage of most opportunities to apply 
the skills appropriately. 



Classroom Implementation Process Data 

Beginning with the onset of training, and continuing through 
the school year, experimental teachers completed a weekly log 
indicating' a)"how many Circle sessions were held during the week, 
b) which students attended each session^ and c) the reasons for any 
absences. Twelve of the fourteen teachers consistently completed 
and returned the weekly log. An average of 23.4 weeks of data (SO = 1.4) 
were collected from these 12 teachers. The other two teachers returned 
only 7 and 13 forms respectively, ignoring msny reminders. 

Some of the teachers deviated from the instructions in completing 
the logs at various times during the year. This .source of error made * 
the data questionable at the student level; hence a weekly class-level 
index of exposure to Magic Circle was constructed. By summing this 
weekly index over tlje number of weeks of data collected' frcm each 
teacher, a conservative estimate was obtained of the total number of 
Circles that the typical student In each class received during the 
year (Circle Quantity). This estimate ranged from 8 tp 36 sessions^ 
and indicated that students in five classes received less than 18 
Circles over the school year. The average class of students partici- 
pated in 22 sessions (SO = 8.7) which amounts to almost one Circle 
(*87) per week over 25 weeks. 

Interviews With Control School Teachers and Ineligible Experimental 
^'clfiool Teachers 

To check for any diffusion of the treatment, interviews were 
conducted in April » 1979 with four principals and three teachers at 
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control J5chools> and with one nonparticipating principal and three 

teachers at the experimental schools. The interviews were conducted 
by a member of the research team> and, were semi-structured and open- 
ended. 

Only one of the control school principals and none of the control 
school teachers had more than a vague understanding of Magic Circle's 
purpose and methods. The one principal who could describe the strategy 
had recently learned about it in a discussion with the Magic Circle^'^''* 
trainer. Among the nonparticipating principal and teachers from the 
experimental schools^ only the principal had any substantive knowledge 
of the strategy^ and he had learned what he knew from attending the 

second in-service training session. 

In Summary^ when interviewed near the end of the school year> the 
control and nonparticipating teachers and 'administrators knew little 
or nothing about Magic Circle's goal's or techniques, so that diffusion 
of the treatment was not a plausible threat to the internal validity 
of the evaluation design. 
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RESULt\S 



A bi^level analysis approach was ^dopted for several reasons. 
The experimental design employed in this. study involved randomly 
assigning schools .{and not classes or students) to the experimental 
or control condition. Least-squares analysis, techniques assume the 
statistical independence of observations. Treati^ig individual students' 
responses as independent when they share a common school and class*^ 
room environment (and common exposure to treatment) is problematic, 
because the likelthood of spuriously obtaining a si9n1ficant treat- 
inent effect Is enhanced* In addition, student datajand classroom 
aggregate data sometimes possess different^; subtenti^e meaning and 
may be subject to different processes resulting in ^different findings. 
Thus, to complonent the student-1«vel analyses, we/also conducted 
class-level analyses of the student data. Me cho$e to examine class* 
level rather than school-level data because student Interdependence 
occurs prlmarny within classes and not within^chools. In addition, • 
teachers werfr- responsible for the implementatfon of the treatment, 
and they^are assigned to particular classrooms. 
Trjeatmmt of Hissing Data 

The problem of missing observations arose at . two points In this 
analysis. In each instance, we adopted a strategy that allowed maxi- 
mum use of the available data points while minimizing outccHne bias. 
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Computation of Scale Scores . A scale score v/as compdted for a 
student if at least 60S£ of the items cotnprising that scale were present. 
Any missing item score was replaced by the mean for that item in the . 
appropriate;subclass of the experimental design. This procedure utilized 
most of the item data and provided unbiased cell means. However, it 
constrained cell variances and inflated degrees of freedom artificially. 
When less than 40% of the items comprising a scale were missing, the 
student received a missing value for that scale. 

Missing Scale Scores in Multivartate Analyses . Each of the 
multivariate analyses in this report was computed from suwnary data 
(i.e.» subclass means and frequencies and a pooled within-class 
variance^covariiince matrix). Choice of this input option enabled 
better utilization of data than typically is possible with available 
statistical routines, which exclude a case with even a single missing 
datum from analysis (c.f., MULTIVARIAHCE: Finn, 1976; SPSS: Cohen 
and Burns, Note 7). Specifically, the subclass mean and variance 
for a variable were based on all cases for which the variable was 
present, and the computation of each subclass covariance included 
every case in which the two variables were present. Hence, the number 
of observations contributing to covariances differed slightly within 
each subclass aatrix. To compensate for our earlier inflation of 
degrees of freedom, we chose the minimum pairwise frequency to repre- 
sent each s,ubclas$ both in computing the pooled matrix and later for 
significance testing. 
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Analysis of Initial Equivalence and. Attrition 

' An analysis was conducted to determine a] whether students in 
the experimental -and control conditions were equivalent on the pre- 
test measures and b) whether attrition had a differential effect on 
these conditions. Initial nonequivalehce between treatment conditions 
has implications for both internal validity (the ability to make ^ 
causal inferences about the effect of the treatment) and external 
validity (the ability to generalize findings to other populations). 
Attrition may affect internal validity if students missing from one 
condition differ systematically in kind from those missing from another 
conditlon^ Attrition may also affect external validity if the missing- 
(attrited) students differ systematically from those who remain in the. 
sample {non«attr1ted). 

As suggested by Ours and g^lass il971), a two-way multivariate 
analysis of variance (MANOVA) j^as peV^ormed pn the student pretest 
data. The factors of the design were treatment condition (experimental 
versus control), and attrition status (attrited versus non-attrited). 
A main effect for treatment (fondition would provide evidence for initial 
non-equivalence. A main effect for attrition status would provide 
evidence for limited external validity. The interaction between treat- 
ment condition and attrition status would indicate a threat to internal 
^validity. ^ / 

None of the tested effects was significant. The respective £-ratios 
for treatment condition, attrition status, and the interaction term 

* . 

34 



were 0.91, 1.60. and 1.56, w.ith'5 and 414 degrees of freedmi. Hence, 
the experimental and contrbl students were initially equivalent, and 
attrition did not impair this equivalence. 
Multivariate Analyses Student Data 

So5ie'T)rior research has indicated that the pffect-of Magic Circle 
may be a function of the students' prior achievement lev&l. A 
preliminary analysis was performed to explore this hypothesis by * 
subjecting the post-test data to MANOVA employing trea6nent con- 
dition (T), sex of student (S), and prior reading level (P), as factors 
in the design. Prior reading level was useti as ari indicator of prior 
achievement, ^his variable was obtained by trichotomizing Prior Read. 
TheVesults of this analysis indicated that'neither the T x P inter- 
action, F{28,690) = 1.18, nor the T x S'x P .interaction, £(28,690) = 
0.99, was significant. Thus, no support was found for the hypothesis 
that students' prior reading level was related to the effects of 
Magic Circle. Hence, the design employed in subsequent analyses 
includes only treatment condition ^and sex (of student) as factors. 

For the multivariate analysis of the student data, a complete 
factorial design was employed for hypothesis- testing, with treatment 
condition and sex as the factors. Multivariate analysis of xovariance 
(MANCOVA) was performed using pretest measures as covariates when the 
appropriate assumptions were inet; otherwise, MANOVA was performed 
on post-test outcomes. Since the design was nonorthogonal due to ^ 
unequal cell sizes, the reported mafn effect^for each factor wq$ 
tested after the variance associated with the other factor had 
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been removed and thfe treatment x sex interefction was tested after 
removing the main "effect variance. Effects were estimated subse- . 
quently using a reduced-rank model based upon the pirior hypothesis 
testing* When significant interactions were obtained, the. simple 
■ treatment effects were tested by employing the Bonferroni t^ statistic 
(Myers, .T579) This statistic is a t-test that controls the Type I 
- error rate for the number of tests performed. Because of space limita-, 
tions, only results -pertaining to the treatment condition and treatment 
Interaction are discussed, 

^ The post-test measures were divided into four subsets based upon 
a priori hypoth^eses about their interrelationships and empirical 
findings concerning. the homogeneity of their covariance structures. 
One subset. (A) consisted of attitudes toward one's social self and 
social relationships; social self-esteem (Social Self) and attitudes 
toward peers (Att Peers), V . 

A second subset (B) consisted of attitudes and behaviors related 
to one's academic self: locus of control; success (Control Succ), 
locus of* control : failure (Control Fail); affective teaching climate 
(Affec Climate); academic self-esteem (Acad Self); at^titudes toward 
' school (Att School); perceived peer attitudes tovard school (Peer 
Att Sch); reading achievement (Rea^d); and mathematics ach levenent 
(Math). A third subset (C) included the two attendance measures, 
unexcused absences (Unexc.Abs) and total absences (Total Abs), and 
the final subset (D) contained the teacher ratings of student minor 
(Minor) pnd majOr (Major) misbehavior. 
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Each of the four subsets of post-test measures was analyzed 
separately; Prelimlnairy analyses were perfornied to identify the 
set of pretest outcomes that best predicted each set of post*test 
outcomes and did not violate the assumptions of the covariance 
analysis (priimarily, that the regression hyperplanes be homogeneous 
across the cells of the design), The final sets of covariates 
selected for the analyses Included the following pretest measures for 
set A: Social Self and Att Peers; and for set 8: Social Self, 
Att Peers, Affec Climate, School Att, and Prior Read* For sets 
C and D, covariate 6elect1on was abandoned because either the pretest 
measures were not good predictors of the post*test outcomes or . 
the hbmogeneity assumptions were violated* For each of these sets 
of measures the KANOVA result^ have been reported. 

■Results of the multivariate analyses of the student data have 
been surrtnarized in Table 5, ^ For Set A, a marginally significant 
treatanent main effect, F^(2,384) - 2.80, £ <,07, and treatment x 
sex interaction, £(2,384) ^ 2.84, £ <.06, were obtained* For 
Set D, a highly significant treatment main effect, F_(2,411) = 19,46, 
£< .0001, and treatment x sex interaction, £(2,411) - 8;54, £ < 
.0003, were found. No treajtment-related effects were significant in 
Sets B and 

The only reliable univariate effects on Set A measures 
occurred for the treatment main effect, F (1 ,385) = 5.23, p< -03, 
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Table .5 

summary of multivariate analyses of student data 

Subset of Multivariate 
Source of Variation Measures df F £<_ 

Treatment Condition (T) A^ 2,384 2.80 .07 

8,371 1 .40 ns 

2,458 0.04 ns 

D^ 2,411 19.46 .0001 

Sex (S) A 2,384 1.90 * ns 

8 8,371 , 3.15 .002 

C 2,458 1.42 ns 

D 2,411 24.55 .0001 

T X S A ,2,384 - '2.84 .06 

B 8,371 1.04 ns 

C 2,458 0.59 ns 

D 2,411 8.54 .0003 

*^Subset A analysis: Variates and covariates were Social Self and Att Peers. 

^Subset B analysis: Variates were Control Succ, Control Fail, Affec Climate, 

Acad Self, School Att, Peer Att Sch, Read, and Math. 
■Covariates. were Social Self, Att Peers, Affec Climate, 
School Att, and Prior Read. 

^Sutjset C analysis: Variates were Unexc Abs and Total Abs* 

Subset 0 analysis: Variates were Minor and Major. 
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and the treatment x ^ex interaction, F(l,385) = 5.17, £< ,03; 
on Social Self, Examination of the simple treatment effects by 
sex yielded a significant effect for boys, t^(385) = 3.12, 
£< .01, but not for girls, t(385) = 0,03. Inspection of the 
means revealed that experimental boys scored .45 standard deviations 
higher than control boys on the pretest- adjusted measure of socjial 
self-esteem. 

From Set D, reliable univariate effects were found on both 
measures. Minor and Major. The respective treatment main effects 
were £(1,412) = 16.45, £ < .0001, and £(1,412) = 6.11, £ < .P2. 
The Treatment x Sex interactions were F{1,412) = 4.28, £< .04, 
and £(1,412) = 5.30, £< .03, respectively. Examination of the - 

simple treatment effects by sex revealed no reliable differences 
for girls on Minor, t (412) * 1.51, or on Major, t (412) = 0.20. 
For boys, however, a negative treatment effect was obtained on 
Minor, t (412) = 4.30, £< .002, and a positive effect on 
Major, t (412) = 3.38, £< .002. ExpeHmental boys were generally 
rated 0.61 standard deviations higher than control boys on the 
minor misbehavior measure and 0.48 standard deviations lower ^ 
than control boys on the major misbehavior measure. 
Analyses of Class-Level Student Data 

For the class-level student analysis each class was divided into 
* two analysis units, one consisting of the boys in the class and one 
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consisting of the girls. For each such unit, mean scores were com- 
puted on the student measures, and separate analyses were conducted 
for the boys arid girls* With this approach^ unit*' of analysis are 
more interdependent than in a strictly class-level analysis, but the 
approach controlsfor^ and enables examination of, the influence of 
student sex upon the treatment effects. 

Univariate analyses of variance were applied to each pretest 
measure with treatment condition as the independent variable. 
The results indicated no initial differences between the treatment 
and control groups for either the boys oi^the girls* Since no initial 
differences wero obtained, a one-way treatment condition univariate 
analysis of variance was performed on eact\ p'^st-test measure. The 
results of these analyses are suinnarized in Table 6. For the boys 
the class-level results were consistent with the student-level results* 
SignifTcant treatment^ffects were obtained on two measures. Social 
Self, F(l, 26) « 16*19, £ < .001 , and Minor, f(l ,24) = 5.1&, £ < .04, 
and a marginally significant effect was obtained on a third. Major, 
F(l,24) « 3.07, £< .09, Congruent with the student-level findings, 
positive treatment effects were obtained on Social Self and Major, and 
a negative treatment effect on Minor. 

For the girls the class-level results were consistent with the 
student-level results as no significant treatment effects were obtaineci* 
Analyses of Teacher Data 

A one-way treatment condition univariate analysis of covariance 
was applied to eac post-test teacher measure with the corresponding 
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Table 5 



SUMMARY OF UNIVARIATE ANALYSES OF CLASS-LEVEL STUDENT D^TA 



Male Classes 



Fonale Classes 



Measure 


df 


F 


JEJl 


df 


F 




Social Self 


1,26 


16.19 


.001 


1,26 


0.02 


ns 


Att Peers 


" 1 ,26 


1 .76 


ns 


1 ,26 


0.30 


ns 


Affec Climate 


1 , 26 


0.00 


ns 


•1,26 


2.13 


ns 


Acad Self 


1,26 


0,83 


ns 


1 ,26 


1.73 


ns 


Control Succ , 


1 ,26 


0.03 


ns 


1,26 


0.58 


ns 


cojitroi Fail 


1 ,26 


0.55 


ns 


It CO 


ri 1/1 
U. 14 


ns 


Att Sch 


1,26 


1.07 


ns 


1,26 


0.15 


ns 


Peer Att Sch 


1,26 


0.01 


ns 


1,26 


0.87 


ns 


Read 

• 


1,26 


0.35 


ns 


1,26 


0.04 


ns 


Math 


1,26 


1.14 


ns 


1,26 


0.06 

c? 


ns 


Unexc Abs 


1,26 


043 


ns 


1,26 


0.20 


ns 


Total Abs 


1,26 


0.15 


ns 


1,26 


0.73 


ns 


Minor 


1,24 


5.16 


.04 


1,24 


2.26 


ns 


Major 


1,24 


3.07 


.09 


1,24 


0.05. 


ns 
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pretest used as a covariate. Since the corresponding pretest did not 
exist for MC Skill, analysis of variance was performed on this measure, 

Results of the univariate analyses of variance and covariance 
on the teacher data are sumnarized in Table 7. Significant treatment . 
effects were obtained on two of the four teacher Measures, Teacher 
Sati^, F(l,21) =^ 10.13, £ < .005 and MC Skill., £(1,24) ^ 11.50, 
£< .003, Experimental teachers generally rated themselves as more 
satisfied with teaching on the post-test {adjusted for their pretest 
scores) than did control teachers. Experimental teachers also scored 
higher than controls on the behavioral inventory reflecting classroom 
implementation of Magic Circle-related skills. 
The Relationship of Student Outcomes to Treatment Implementation 

The analyses reported thus far involved comparisons of treated 
versus untreated students, teachers, and classrooms. Just as ^hese 
groups may differ on the basis of presence or absence of the Magic 
Circle treatment, the experimental classes might also differ as a 
function of the '^amount*' of treatment received. Therefore, exploratory 
analyses were conducted to investigafte the relationship among the 
experimental classrooms* average student post-test outcomes and the 
classrooms* amount of exposure to Magic Circle. Classroom exposure 
to Magic Circle (Circle Quantity) was measured by an index constructed 
from the teachers* weekly reports described earlier in the section on process 
evaluation. This measure was dichotomized at the median resulting in 
two groups of experimental classrooms, a high implementation group and 
a low implementation group.^ The high implementation group received 

^ While we recognize that a median split sacrifices possibly meaning- 
ful data, we were unwilling to assume a linear relationship between imple- 
mentation and student outcomes, an assumption necessary for a correla- 
tional analyris approach. 
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Table 7 

summary of univari/^fe analyses of teacher data 

/ 

/ 

f 

Treatment Main Effect 



Measure 


/ Analysis 








Type a 


F 


df _ 




MC Object 


C 


0.05 


1,21 


ns 


Teacher Satis 


C 


10.13 


1,21 


.005 


Faculty Cohes 


C 


0.92 


1,21 


ns 


MC Skill 


V 


11.50 


1,24 


.003 



^C « analysis of covariance 
V * analysis of variance 
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an average of 28,7 Circles (SD = 3.8) > and the low implenentation group 
averaged 14.9 Circles (SD = 6.3) over the school year. 

Analyses of variance were performed on the class mean student 
pretest data to determine whether high and low implementation classes 
initially differed on these measures. The results indicated no inT*tial 

dif-ferenees-^^r the-boys^- The girls differe<l on one pretest- me&5ure-, 

Prior Read, £(1,12) = 3.84, .08, with the high implementation group 
initially having higher reading scores. 

For the boys, since no initial differences were obtained, uni- 
variate analyses of variance were performed on the class mean student 
post-test data with implementation level (high versus low implementation) 
as the independent variabTe.\ Results from these analyses are sumnarized 
in Table 8. No significant differences were obtained; that is, the 
level of treatment implementation did not have a detectable effect 
upon any student measure. Of course, the statistical power of these 
analyses was limited due to the small number of classes that were available 

For girls, univariate analyses of covariance were performed on the 
class mean student post-test data with implementation level as the 
independent variable and Prior Read as the covariate. These analyses 
partially control fo" the initial differences between the groups. On 
six of the measures, analyses of variance of the post- test data were 
reported as the assumptions of the covariance analyses were not met. 
The results have been summarized in Table 8. A significant difference 
was obtained, on one of the fourteen measures, Unexc Abs, F(l,ll) ^ 5.34, ^ 
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Tabls 8 

summary of treatment implementation analyses 





Male Classes 


Female Classes 


Measure 




P < 
ns 


F(l,ll) 




Social Self 


0.72 


1.46^^' - 


ns 


Att Peers 


0.02 


ns 


0.07 


ns 


Affec Clfmate 


0.74 


ns 


1.75^ 


ns 


Acad Self 


0.28 


hs 


0.55 


ns 


Control Succ 


Q.21 


ns 


0.00 


ns 




0.76 


ns 


0.05 


ns 


Att Sch 


"0.28 


..IS 


0.14^ . 


ns 


Peer Att Sch 


3.02 . 


ns 


1.87 


n^ 


Read 


0.71 


ns 


0.30^^ 


ns 




2.81 


ns 


0.04 


ns 


,Unexc-Abs 


0.66 


ns 


5.34 


.05 


'Total Abs 


' 3.12 


ns 


O.Ol"" 


ns' 


Minor 


, 0.00 


ns 


0.06 


ns 


Major 


. 0.62 


ns 


1.22« 


ns 



'tote: The independent variable Is treatment implementation level (high versus low). 
For the males, analyses of 'variance of the class mean data have been reported 
For the females, .analyses of covarlance of the class mean data using Prior 
Read as a covarlate have been reported (see superscript). - 

^Analysis of variance vras performed* F{1,12), due to violations of covarlance 
assumptions. ' . 
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p < ,05. Girls 1n the high implementation group had fewer unexcused 
absences than those In the low implementation group. Given the total 
number of significance tests performed In these analyses, it Is likely 
that this single observed difference may be a Type I error. 
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Discussion 



process idata indicated that teachsrs in the experimental 
grou^ found the Magic Circle In-service training to be Interesting, 



well organized, and useful. Furthermore, the teachers reported that 
they had mastered most of the skills taught in the training and had 
Implemented) them regularly In their classrooms. The only weakness 
of^ the training reported by a few teachers was that they found the 
last few training sessions to be redundant. 

The frequency wIthNihich teachers conducted Circles in their 
classroom varied consfd&rably. Over the school year, some classes 
received one Circle every^hree weeks, whgpeerTTJther classes received 
at least one Circle per week^ In spne of this variation In frequency 
of Implementation, the quality of the'teachersVtircle leadership, as 
rated by the trainer, was uniformly high. This observational finding 
coincided with the teachers' self-reported mastery of the skills. 

Consistent with the process results, on post-testing, experimental 
teachers rfeported using Circle-related skills in their classrooms 
to a greater extent that did control teachers. Experimental teachers 
also reported greater personal Satisfaction with teaching than did 

control teachers (after adjusting for pretest levels of satisfaction), 

■i> 

Thus, teachers who participated In the training'seem to have benefited 
from the experience. 
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As compared with controls, boys in the experimental group generally 
were rated by their teachers to be major discipline problems to a lesser 
extent, but minOr discipline problems to a greater extent. This finding 
does not coincide with the teacher Interview data since experimental 
teachers reported only positive effects of Magic Circle upon student 
classroom behavior. This pattern of results suggests that experimental 
teachers may have used the two discipline rating scales in a different 
manner than did the control teachers. The experimental teachers may 
have employed different definitions' of "minor and major discipline 
problems.** 5uch ^ cognitive bias in rating could possibly be attributed 
to the Magic Circle training or the experience- of leading Circles In 
the classroom. However, the bias Is just as likely to be motivational 
as cognitive, and may be attributable to self-enhancing, self-defensive 
Or positlvity biases, or due to social desirability or compensatory 
rivalry (Cook and Campbell, 1979). As the validity of these teacher' 
ratings of student irisbehavior Is questionable, so Is the pattern of 
results obtained from them. 

As hypothesizecl. Magic Circle Increased boys' social self-esteem 
relative to controls. This effect was, however, somewhat puzzling 
since It obtained only for boys, and not for girls, and boys and girls 
were Initially equivalent on this measure. In addition, it was an iso- 
lated effect in that boys' attitudes toward peers, which conceptually 
and empirically are related to their social self-esteem were not 

^We had anticipated this possibility and Included a stnjctured 
questionnaire with the Student Behavior Report requesting the teachers 
to classify 17 different hypothetical misbehaviors as minor, major. 
Or not discipline problems. Unfortunately, most of the teachers mis- 
interpreted the Instructions, rendering these data useless. 
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affected by Magic CIrcTifv- Furthermore, improved social self-esteem 
among experimental boys showed no relatlonshl^to the measure of 
treatment implementation, Circle Quantity. Thus\ the increased 
social self-esteeiTi observed among experimental boy'i^cannot be explained 
by the number of Circles In which they participated! In fact, none of 

^ the boys* ou tcomjes was reliably relat ed to Imple mentation . Thes e 

findings suggest that even minimal participation in Magic Circle may be 
sufficient to Improve boys'. social self-esteem. While this Is not a 
satisfying explanation, it Is .the only hypothesis Which the data do 
not contradict. A number of alternative hypotheses have been explored 
through secondary analyses of the data. These have not been described 
In this paper, and none was found to be tenable. 

The student outcome data showed no effects of Magic Circle on girls 
In the experimental group as compared to control group gtrls. In addition, 
among the experimental classes, no relationship was found between the ' 
level of treatment Implementation and post-test outcomes. 

Students In the present study may have received considerably fewer 
Circles than did students In previous studies, although our method of 
estimating level of implementation was decidedly conservative. By our 
estimation method, students received an average of 22 Circles, whereas 
the average in 13 prior studies, as reported In the HOTI review, was 
60 Circles. We do not know how Impl mentation data were collected In 
other studies, and it may be that some of them were biased in a liberal 
direction. Several of these studies Involved very small numbers of 
students and teachers, and the implementation levels achieved In them 
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may be atypical. Also, in several studies, "outside" personnel rather 
than classroom teachers were used to conduct Circles, Given 'the numbers 
fnvolved in the present study, and that several efforts were made to 
stimulate voluntary implementation by the teachers, the obtained levels 
may represent a more realistic approximation of what can pe achieved 
thro\igh" voluntary ~fnvo^ reguTar~classroom teachers. 

It is also noteworthy that in prior studies cited by HDTI, no 
reljationship existed between implementation level and student outcomes. 
Thus, the plausible expectation that greater Implementation will produce 
more positive treatment effects has not been confirmed as yet. 

In the present study, although a strong research design was 

utilized and many types of process and outcome data were collected, 

the pattern of results was largely unclear. Despite various incon- 

sistencles, threBf general conclusions seem wat^ranted: 

1- Teachers who participated in the Magic Circle In-service 
training found it Interesting and useful. They mastered 
most of the Magic Circle skills and Impletnertted them in 
their classroom to'varying degrees. Their personal satis- 
faction with teaching Improved more than the control teachers, 

2, In comparisons of the experimental and control group, Magic 
Circle was found to have little detectable positive effect 
upon the student outcomes examined. The one notable effect 
was upon boys* social self-esteem. 

3. The relationship between the treatment process and Its effect 
upon the experimental boys appears to be complex and may be 

a function of process variables not measuMd In the current 
study (e.g., the specific curriculum employed by the teachers 
In the Circle sessions) . 

Resolution of this puzzling pattern of results may come with continued 

delivery of services to the experimental cohort during the next two years. 
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students in the experimental group are scheduled to receive a second 
year of Magic Circle via their fourth grade teachers, and other inter- 
ventions via their fifth grade teachers. Thus, if the inconsistent 
pattern of results is a function of the limited duration and intensity 
pf services delivered to date, the second and third years of this 
study may show a more coherent and decisive set of effects. 
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